Smart de-fragmentation of file systems inside VMS for fast rehydration in the cloud and efficient deduplication to the cloud

ABSTRACT

One example method includes chunking a respective disk of each of a plurality of virtual machines (VM) to create a respective plurality of chunks associated with each of the VMs, creating, based on the chunking process, a cluster comprising one or more of the VMs, creating a VM template whose data and disk structure match respective data and disk structures of each of the VMs in the cluster, and in response to a file operation involving a first one of the VM disks, defragmenting the first VM disk so that a disk structure of the first VM disk is the same as a disk structure of the VM template.

FIELD OF THE INVENTION

Embodiments of the present invention generally relate to dataprotection. More particularly, at least some embodiments of theinvention relate to systems, hardware, software, computer-readablemedia, and methods for providing cloud DR protection for VMs which havefilesystems that are logically similar or identical, but are physicallystored in different ways on disk.

BACKGROUND

Enterprises may generate significant amounts of important data that istypically preserved in some type of data protection environment. Suchdata protection environments help to ensure that the data is recoverablein the event that a disaster occurs that compromises, or prevents accessto, the data. Thus, disaster recovery (DR) is a critical process fororganizations.

In general, a DR to cloud system allows users to failover or to performa DR test of virtual machines from on premise environment to a publiccloud environment. To be efficient and save costs, the data is stored ina relatively cheap object storage in the public cloud and is restored toa block storage upon demand only, when a failover or DR test areinitiated. The system also does not keep the data in a de-duplicatedform due to the large amount of metadata needed and the slowdownexpected in the rebuild of the virtual machines (VM), since the rebuildin such case will require reading data from a significantly largeramount of objects and, thus, the total cost of ownership (TCO) will behigher.

Disaster recovery plans for VMs present some particular challenges. Forexample, the VMs in a group may have filesystems (FS) that are almostidentical logically, but the respective filesystems may be physicallystored in a very different way on the disk. Further, some processes maychange the location of files on a file system of a VM, such that VMsthat are otherwise nearly identical may have substantial difference intheir respective VMDK binary images. Moreover, files deleted from a VMfilesystem are not zeroed in many cases. Thus, even if most of the VMfilesystem is empty, there may still be a large amount of garbage on thedisk. These circumstances are problematic.

For example, when replicating a VM to the cloud for DR purposes, andthen restoring multiple, almost identical, VMs, it may be difficult orimpossible to simply use a single block device as a backup template, andthen keep only the differentials relative to each VM. As well, if thereis a process that changes a VM filesystem structure without actuallychanging files, that process may cause large unnecessary differencesbetween two versions of the VM that are otherwise practically identical.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which at least some of the advantagesand features of the invention can be obtained, a more particulardescription of embodiments of the invention will be rendered byreference to specific embodiments thereof which are illustrated in theappended drawings. Understanding that these drawings depict only typicalembodiments of the invention and are not therefore to be considered tobe limiting of its scope, embodiments of the invention will be describedand explained with additional specificity and detail through the use ofthe accompanying drawings.

FIG. 1 discloses aspects of an example operating environment for someembodiments of the invention.

FIG. 2 discloses aspects of an example host configuration.

FIG. 2 discloses aspects of an example method for creating a VMtemplate.

FIG. 4 discloses aspects of a method for defragmenting a VM disk.

FIG. 5 discloses aspects of a method for restoring one or more VMs usinga VM template.

DETAILED DESCRIPTION OF SOME EXAMPLE EMBODIMENTS

Embodiments of the present invention generally relate to dataprotection. More particularly, at least some embodiments of theinvention relate to systems, hardware, software, computer-readablemedia, and methods for providing cloud DR protection for VMs which havefilesystems that are logically similar or identical, but are physicallystored in different ways on disk. At least some embodiments of theinvention are implemented in connection with a production site thatincludes one or more VMs, and a cloud storage environment, such as acloud datacenter for example, although the scope of the invention is notlimited to this example environment. In general, embodiments of theinvention include a method which will allow the disk layout of similarVMs to remain as similar as possible after changes, allowing customersto save cloud storage space and keep small amount of templates in thecloud, as well as enable faster recovery times in cloud DR use casessuch as multi-VM failover or DR Test.

In more detail, example embodiments of the invention concern theprovision of cloud DR protection for VMs which are practically similar,with very small differences. For this purpose, a similarity algorithm,and a distributed defragmentation (or ‘defrag’) algorithm, are provided.The similarity algorithm attempts to identify VMs that are similar toeach other in terms of their respective data and filesystems, and thedistributed defragmentation (or ‘defrag’) algorithm, such as in the formof an agent, uses predefined consistent logic to arrange data on similarVMs and attempt to put identical data at the same offsets of the disk.More particularly, the defragmentation agent is installed on productionsite VM clients that are desired to be protected. The defrag agentplaces the data on the respective VMs such that the VM disks of theprotected machines that have almost identical data will also be almostidentical in their disk layout.

It is important to note that due to file system properties and datamovement, even identical file systems containing the same files anddirectories can have a very different disk structure due to differentplacement of the files on the disk. Thus, even versions of the samevirtual machine which logically differ very little from each other mayhave significantly different disk binary layout.

In more detail, the defrag agent installed on each production site VMwill perform a variety of functions with respect to empty disk space,and with respect to the data on the disk. For example, the defrag agentwill operate such that when a file is deleted, the defrag agent willzero the data, that is, the space left open by deletion of the file isfilled with zeros, and these zero areas will then be moved to the end ofthe disk.

As well, the defrag agents will, to the extent possible, place identicaldata in different, but similar, VMs in the same location on therespective VM disks. To this end, the similarity algorithm will providea sample VM for a cluster of VMs, and the defrag agents operate toensure that the data of the other VMs will be arranged in a similar wayas in the sample VM. For example, if in a new version of the VM, someprocess just moved data in the file system, such as by rewriting a filewith a small change for example, to a new place, the defrag agent willmove the data to the same location as it was before the overwrite of thefile.

Advantageously then, some embodiments of the invention may providevarious benefits and improvements relative to the configuration andoperation of conventional hardware, software, systems and methods. Toillustrate, an embodiment of the invention includes processes forestablishing and maintaining, as much as possible, a common diskconfiguration among a population of VMs that are to be protected.Further, an embodiment of the invention provides a single VM recoverytemplate, which may be in the form of a full VM image that is stored inblock storage at a cloud site, that can be used to restore similar VMsof the population of VMs. Because only a single VM recovery template isstored, a relatively small amount of block storage is employed ascompared with an arrangement where multiple recovery images are used,thereby reducing costs since block storage tends to be more expensivethan, for example, object storage. An embodiment of the inventionprovides for relatively fast DR processes, since a single VM recoverytemplate is used to restore multiple, similar, VMs.

It should be noted that the foregoing advantageous aspects of variousembodiments are presented only by way of example, and various otheradvantageous aspects of example embodiments of the invention will beapparent from this disclosure. It is further noted that it is notnecessary that any embodiment implement or enable any of suchadvantageous aspects disclosed herein.

A. Aspects of an Example Operating Environment and Operations

The following is a discussion of aspects of example operatingenvironments, and associated operations, for various embodiments of theinvention. This discussion is not intended to limit the scope of theinvention, or the applicability of the embodiments, in any way.

In general, embodiments of the invention may be implemented inconnection with systems, software, and components, that individuallyand/or collectively implement, and/or cause the implementation of, datamanagement operations. Such data management operations may include, butare not limited to, data read/write/delete operations, data backupoperations, data restore operations, data cloning operations, dataarchiving operations, and disaster recovery operations. Thus, while thediscussion herein may, in some aspects, be directed to a discussion ofdata protection environments and operations, the scope of the inventionis not so limited. More generally then, the scope of the inventionembraces any operating environment in which the disclosed concepts maybe useful. By way of illustration, but not limitation, embodiments ofthe invention may be employed in connection with data backup and restoreplatforms such as the Dell-EMC NetWorker and Avamar platforms, as wellas the Dell-EMC RecoverPoint and the VMWare SRM (Site Recovery Manager)platforms.

A data protection environment may take the form of a public or privatecloud storage environment, a customer on-premises storage environment,and hybrid storage environments that include public and privateelements, although the scope of the invention extends to any other typeof data protection environment as well. Any of these example storageenvironments, may be partly, or completely, virtualized. The storageenvironment may comprise, or consist of, a datacenter which is operableto service read and write operations initiated by one or more clients,and which performs disaster recovery (DR) operations including creatingand writing to recovery disks.

In addition to the storage environment, the operating environment mayalso include one or more host devices, such as clients for example, thateach host one or more applications. As such, a particular client mayemploy, or otherwise be associated with, one or more instances of eachof one or more applications. In general, the applications employed bythe clients are not limited to any particular functionality or type offunctionality. Some example applications and data include emailapplications such as MS Exchange, filesystems, as well as databases suchas Oracle databases, and SQL Server databases, for example. Theapplications on the clients may generate new and/or modified data thatis desired to be protected.

Any of the devices or entities disclosed herein may be protected by oneor more data protection policies according to various embodiments of theinvention. Yet other examples of devices that may be protected by way ofa data protection policy according to embodiments of the inventioninclude, but are not limited to, containers, and VMs.

Any of the devices, including the clients, servers and hosts, in theoperating environment can take the form of software, physical machines,or virtual machines (VM), or any combination of these, though noparticular device implementation or configuration is required for anyembodiment. Similarly, data protection system components such asdatabases, storage servers, storage volumes (LUNs), storage disks,replication services, backup servers, restore servers, backup clients,and restore clients, for example, can likewise take the form ofsoftware, physical machines or virtual machines (VM), though noparticular component implementation is required for any embodiment.Where VMs are employed, a hypervisor or other virtual machine monitor(VMM) can be employed to create and control the VMs.

As used herein, the term ‘data’ is intended to be broad in scope. Thus,that term embraces, by way of example and not limitation, data segments(or simply ‘segments’) such as may be produced by data streamsegmentation processes, data chunks, data blocks, atomic data, emails,objects of any type, files, contacts, directories, sub-directories,volumes, and any group of one or more of the foregoing.

Example embodiments of the invention are applicable to any systemcapable of storing and handling various types of objects, in analog,digital, or other form. Although terms such as document, file, block, orobject may be used by way of example, the principles of the disclosureare not limited to any particular form of representing and storing dataor other information. Rather, such principles are equally applicable toany object capable of representing information.

With particular attention now to FIG. 1 , an operating environment 100may comprise, or consist of, a data protection environment. The dataprotection environment can include an enterprise datacenter, or a clouddatacenter, or both. The data protection environment may support variousdata protection processes, including data replication, datadeduplication, cloning, data backup, and data restoration, for example.As used herein, the term backups is intended to be construed broadly andincludes, but is not limited to, partial backups, incremental backups,full backups, clones, snapshots, continuous replication, and any othertype of copies of data, and any combination of the foregoing. Any of theforegoing may, or may not, be deduplicated.

In general, the example operating environment 100 in FIG. 1 comprises aproduction site 200, and a datacenter 300, which can be a cloud storagedatacenter, although that is not necessarily required. The productionsite 200, which may take the form of an enterprise or organizationpremises for example, may include any number ‘n’ of production VMs 202.As well, the datacenter 300 may, but is not required to, be a publiccloud site such as Amazon AWS with Amazon Elastic Block Store (AmazonEBS), Microsoft Azure, or Google Cloud.

It is noted that as used herein, the term ‘VM’ is broad in scope andrefers generally to a virtualized representation of any combination ofhardware and/or software. The disclosed VMs are not limited toperformance of any particular functionality or group of functionalities.Some example VMs in connection with which embodiments of the inventionmay be implemented are virtualizations of components that include, butare not limited to, file systems, applications, databases, disks,processors, storage elements, memory elements, software, hardware, andany combination of these.

In general, the production VMs 202 may be involved in operations ofcomputing systems and other systems at the production site 200, and theproduction VMs 202 may collectively form a cluster. One of theproduction VMs, designated as VM1 204 in FIG. 1 , may serve as atemplate or sample for configuring the respective filesystem and/or dataof the other production VMs 202. In some embodiments, a sample VM 204 isprovided that is not a production VM.

The production site 200, or alternatively the datacenter 300, mayadditionally include a similarity module 206. In general, the similaritymodule 206 identifies those production VMs 202 that are similar to eachother in terms of, for example, their respective data, and/or disklayouts. The similarity module 206 can be a stand-alone module, althoughthat is not necessarily required. In terms of its operation, thesimilarity module 206 may query or poll a group of production VMs 202,either automatically or at the initiative of a user such as anadministrator, to ascertain whether, and where, any such similarityexists between those production VMs 202. The extent to which a group ofproduction VMs 202 have sufficiently similar data and disk layouts as tobe designated or deemed by the similarity module 206 as being ‘similar’to each other need not be fixed, although it can be, and can be definedas needed based on the requirements of a particular application oroperating environment, for example. For example, an administrator orother user can define what constitutes adequate similarity between twoor more production VMs 202.

There are a variety of ways to determine or decide whether or notproduction VMs 202 are similar to each other. One area to check inmaking a determination as to similarity is the different backup copiesof the same VM. However, in some cases at least, it may be that most ofthe data will be placed at the same disk locations on the otherproduction VMs 202. Another way to evaluate similarity is to have a usermanually indicate which VMs are created by a template. In some operatingenvironments, this data can also be retrieved using management toolslike Dell-EMC VMware vRealize Automation (vRA) for example. Thisapproach can be effective where there are also VMs which are verysimilar to each other due to their creation by a template, for example,user VMs, where the customer created all the VMs with similar software.

As well, the similarity module 206 may comprise, or consist of, aclustering algorithm, embodied as a clustering module at the productionsite 202 for example, that can be employed in connection with thechunking of the disks of the production site VMs 202. The clusteringmodule may comprise, or consist of, the similarity module 208 and achunk module 208.

For example, the production site 200, or alternatively the data center300, may include the chunk module 208, which serves to chunk the disksof the production VMs 202 to relatively large pieces, such as 4 MB forexample. Then, for each production VM 202, the chunk module 208 createsa vector of hashes, and runs a process which will cluster the productionVMs 202 into several groups. In particular, this clustering process canbe performed using a clustering algorithm such as the k-means clusteringalgorithm for example, where the distance between two or more productionVMs 202, that is the relative similarity of those VMs 202 to each other,is determined by identifying the amount of pieces, or chunks, which arenot common to the production VMs 202 under consideration. Finally, withrespect to the production site 200, and as discussed in more detailbelow, each of the production VMs 202 may include a respective instanceof a defrag agent 202 a.

With continued reference to FIG. 1 , the datacenter 300 serves as the DRtarget in at least some embodiments of the invention. That is, one ormore of the production VMs 202 will be backed up at the datacenter 300.In one particular embodiment, discussed in more detail elsewhere herein,a single production VM 202 that is representative of each of theproduction VMs 202 in a group of production VMs 202, is backed up at thedatacenter 300 and then used to restore, as part of a DR process, theproduction VMs 202 of the group.

The datacenter 300 may include both block storage 302, and objectstorage 304. As noted herein, block storage 302 may be relatively moreexpensive, in financial terms, than object storage 304. A full image 306of a production VM 202, such as VM1 202, can be created and stored inthe object storage 304, as part of a backup process for example. Thefull image 306 is later used to restore the production VM1 202 at thedatacenter 300 as part of a DR process, and the restored VM1 308 is thenused, in connection with any differential VM backups 310, to restore oneor more of the production VMs 202 at the datacenter 300. In the exampleof FIG. 1 , a single restored production VM2 312 is indicated, butmultiple production VMs 202 can be restored at the datacenter 300, basedon the restored VM1 308.

Finally, a recovery algorithm 314 is provided that may be hosted at thedatacenter 300. In other embodiments, the recovery algorithm 314 can behosted at a stand-alone server that communicates with the productionsite 200 and the datacenter 300. In still other embodiments, therecovery algorithm 314 can be hosted at the production site 200. Ingeneral, the recovery algorithm implements and/or causes theimplementation of a DR process for the production VMs 202 by creatingthe restored VM1 308 and then using the restored VM1 308 anddifferential VM backups 310 to restore one or more of the production VMs202 at the datacenter 300.

B. Aspects of an Example Defrag Agent

As noted earlier, each of the production VMs 202 may include arespective instance of a defrag agent 202 a. When present, the templateVM 204 is also provided with a defrag agent 204 a. While no particulararrangement is required, the instances of the defrag agent 202 a can bepushed out to their respective production VMs 202, such as by anadministrator for example. Alternatively, the defrag agent 202 a may bebuilt into the production VMs 202 at the time the production VMs 202 arecreated. As discussed in more detail below, the defrag agent 202 a onthe production VMs 202 will place data on the respective VM disks suchthat the VM disks of the protected machines, that is, the production VMs202, that have almost identical data to each other will also be almostidentical to each other in their respective disk layouts. That is, theexample defrag agent 202 a comprises a distributed defragmentationalgorithm which uses predefined consistent logic to arrange data onsimilar production VMs 202 and attempts to put identical data at thesame offsets of the respective disks of the production VMs 202.

In more detail, the example defrag agents 202 a perform at least thefollowing processes, the first of which concerns empty spaces on a diskof a production VM 202. Particularly, when a file is deleted from aproduction VM 202 disk, the defrag agent 202 a will zero the data, andwill move the zero areas to the end of the disk. That is, the space madeavailable by deletion of the file is filled with zeros, and these zeroareas will then be moved to the end of the production VM 202 disk. Thus,as among a group of similar production VMs 202, not only will the disksof the production VMs 202 have the same data, but that data will bearranged in the same way on each of those disks.

That is, the similarity algorithm will provide a sample VM 204 for thecluster, and the data of the other production VMs 202 will be arrangedin a similar way as in the sample VM 204, and then the defrag agent 202a will try to place identical data in different production VMs 202 inthe same location on the disk. For example, if in a new version of theproduction VM 202, a process just moved data in the file system of thatproduction VM 202, such as by rewriting a file with a small change forexample, to a new location on the disk of the production VM 202, thedefrag agent 202 a will move the data at the same disk location that itwas at before the overwrite of the file took place.

C. Aspects of Example Recovery Processes

With continue reference to the example defrag agent 202 a, it can be seethat leveraging the defrag agent 202 a instances in the production VMs202 which form a cluster, identical disk areas on different productionVMs 202 can be detected and then a single template, such as VM1 204 forexample, can be created, stored, and used to create an EBS, or other,disk at the datacenter 300. When restoring one or more production VMs202, from this single cloud VM1 308, a snapshot is created of the EBS orother disk, and any changes between the snapshot and the productionVM(s) 202 that are needed to be recovered are applied to that disk togenerate the restored VM2(s) 312.

It is noted that embodiments of the invention can employ a template VM204 to restore as few as a single production VM 202. However,identifying a larger set of production VMs 202 which are similar to eachother will enable the use of relatively fewer templates VM1 204 and,thus, fewer full VM images to store in EBS, or the datacenter 300, whichwill in turn reduce the amount of relatively expensive block storagespace 302 needed, and also improve DR recovery times. Because a singletemplate, such as VM1 204 for example, can be used to restore multipleVM2 s 312, thus obviating the need to create and store multipletemplates and/or full VM1 images at the datacenter 300, embodiments ofthe invention are thus effective in implementing data deduplication.This data deduplication provides for financial savings to the customerinsofar as it reduces the amount of block storage 302 consumed at thedatacenter 300.

Turning now to aspects of some example DR recovery processes,embodiments of the invention operate to create a temporary recovery VM1308 with a disk attached to it that is the same size of the VM 202 diskwhich is to be restored. The objects are read from the object storage304, specifically, from the VM1 full image 306, and written to theattached disk. Next, the disk filesystem (FS) is mounted, and drivermodifications to allow the disk to boot on the cloud, such as at thedatacenter 300 for example, are installed. Once the disk is ready, thetemporary recovery VM1 308 is shut down and the disk is attached to anew VM as the boot disk and the new VM is restored as VM2 312. Note thatthe non-boot disk has the same process, but there is no need to installdrivers for that non-boot disk.

D. Example Host and Server Configurations

With reference briefly now to FIG. 2 , any one or more of the componentsdisclosed in FIG. 1 can take the form of, or include, or be implementedon, or hosted by, a physical computing device, one example of which isdenoted at 400. As well, where any of the aforementioned elementscomprise or consist of a virtual machine (VM), that VM may constitute avirtualization of any combination of the physical components disclosedin FIG. 2 .

In the example of FIG. 2 , the physical computing device 400 includes amemory 402 which can include one, some, or all, of random access memory(RAM), non-volatile random access memory (NVRAM) 404, read-only memory(ROM), and persistent memory, one or more hardware processors 406,non-transitory storage media 408, I/O device 410, and data storage 412.One or more of the memory components 402 of the physical computingdevice 400 can take the form of solid state device (SSD) storage. Aswell, one or more applications 414 are provided that comprise executableinstructions.

Such executable instructions can take various forms including, forexample, instructions executable to perform any method or portionthereof disclosed herein, and/or executable by/at any of a storage site,whether on-premise at an enterprise, or a cloud storage site, client,production site, datacenter, backup and restore server, to performfunctions disclosed herein. As well, such instructions may be executableto perform any of the other operations disclosed herein including, butnot limited to, read, write, backup, storage, snapshot, chunking,similarity evaluation, recovery including disaster recovery,deduplication, and restore, operations.

E. Aspects of Some Example Methods

With reference next to FIG. 3 , details are provide concerning methodsfor creating a VM template that can be used as a basis fordefragmentation of the disks of a population of VMs, where one exampleof such a method is denoted generally at 500. The method 500 can, insome embodiments, be cooperatively performed by a chunk module and asimilarity module, examples of which are disclosed herein. Thisfunctional allocation is provided only by way of example however, and isnot intended to limit the scope of the invention in any way.

The method 500 can begin when the respective disks of one or moreproduction VMs are chunked 502, such as by a chunk module for example.The disks can be chunked 502 into any suitably sized pieces. In someembodiments, the chunks are 4 MB in size, but the disks can be chunkedinto bigger, or smaller, pieces. Typically, though not necessarily, eachchunked disk is chunked into pieces that are the same size as the chunksof the other disks.

Next, a vector of hashes of the chunks is created 504 for each of theVMs. A similarity module, for example, then evaluates the vectors andclusters 506 the VMs into one or more similarity groups, based on acomparison of the vectors with each other. In particular, thisclustering process 506 can be performed using a clustering algorithmsuch as the k-means clustering algorithm for example, where the distancebetween two or more production VMs 202, that is the relative similarityof those VMs 202 to each other, is determined by identifying the amountof pieces, or chunks, which are not common to the production VMs 202under consideration. The more chunks that two or more VMs have incommon, the more similar those VMs are.

Once the similar production VMs have been identified, a VM template isthen created 508, such as by a similarity module for example, where thedisk structure and data of the VM template are similar, or identical, tothose of the production VMs. The VM template, like the production VMs,includes an instance of a defrag agent. As changes are made by therespective defrag agents, discussed further in connection with FIG. 4 ,full images of the VM template are taken 510 and backed up 512 at thedatacenter or other data protection environment.

With reference now to FIG. 4 , details are provided concerning someexample methods according to various embodiments of the invention, whereone example method is denoted generally at 600. In general, the examplemethod 600 is concerned with smart defragmentation of a portion, or all,of a VM, although other embodiments of the method 600 could be appliedto other devices, either in whole or in part. In at least someembodiments, the portion of the VM or other device that is defragmentedis part of the filesystem of that VM or other device.

Embodiments of the method exemplified by the method 600 can be performedin whole, or in part, at a production site. In particular embodiments,the method 600 is performed in whole or in part by a defrag agent of aVM, or other device, such as at a production site for example. As well,embodiments of the method exemplified by the method 600 may be performedin connection with a sample or template VM.

The example method 600 may begin when a file operation is detected 602involving a file in the VM filesystem. The file operation may be, forexample, a write or delete operation. In the case of a write operation,the data identified in the write request is written 604 to a VM disklocation specified by the VM template.

When the detected file operation 602 is a file delete operation, thefile data is zeroed 606, and the zeroed areas moved 608 to a particularpart of the VM disk, such as the end of the disk. The particulardestination specified on the VM disk for the zeroed areas can bespecified in the template VM, which may be accessed, for example, by aninstance of a defrag agent residing on the VM where the file deletiontook place. The VM template can be created by a similarity algorithm,examples of which are disclosed herein, and the VM template may resideat a production site along with the VM whose data is being moved.

As noted, the scope of the method 600 is not limited to simply to deleteoperations, but may additionally, or alternatively, apply to file writeand file modification operations which can be detected 602, orconcerning which a notification may be received. To illustrate, if in anew version of the VM, some process just moved data in the filesystem ofthat VM, such as by rewriting a file with a small change for example, toa new place, the defrag agent instance at that VM will move the data tothe same location as it was before the overwrite of the file. Moregenerally, in response to a data modification operation on the VM disk,the defrag agent will move data 608 on the VM disk as indicated by theVM template.

Thus, each of the defrag agent instances at a group of VMs operates to,among other things, defragment the disk of the associated VM, and storecommon data at the same respective locations on each of the VM disks. Inthis way, the defrag agents collectively operate to ensure similar, oridentical, disk structure and data on each of the VMs in a population orcluster of similar VMs. The basis for the disk structure and data to beimplemented on each VM disk is provided by the VM template, accessed bythe defrag agents.

With reference now to FIG. 5 , details are provided concerning examplemethods of using a full image of a VM template to restore one or moreVMs at a target site, such as a cloud datacenter for example. Oneexample of such a method is denoted generally at 700 in FIG. 5 . In atleast some embodiments, part, or all, of the method 700 may be performedat a datacenter, such as a cloud storage site for example, although thatis not necessarily required.

The method 700 can begin with receipt of one or more full images 702 forbackup. The full images may be full images of a VM template, which canbe located at a production site, although that is not required. The fullimages may be received 702 on an ad-hoc, regular, or any other, basis.After the full images have been received, they are stored at thedatacenter 704, in object storage for example.

At some point after storage 704 of at least one full image of a VMtemplate, a disaster recovery (DR) request is received 706. The DRrequest may be generated, and received 706, in connection with a problemat a production site. The problem may be an event, or events, thatcauses corruption or loss of data, such as one or more production VMsfor example. The DR request, which may be received from a productionsite, an administrator, or other entity, may specify that one or moreproduction VMs be restored at the datacenter and/or at another DR targetsite.

After receipt of the DR request 706, a temporary recovery VM is thencreated 708. This temporary recovery VM is created 708 based on a fullimage of a VM template. The full image of the VM template may reside atthe datacenter. In some cases, there may be multiple full images of theVM template at the datacenter, and creation of the recovery VM can bebased on any of those images. The particular full image that is used tocreate the temporary recovery VM 708 may be selected based on aparticular restore point or time identified in the DR request.

After the temporary recovery VM has been created 708, the temporaryrecovery VM is then used to restore 710 the VM(s) identified in the DRrequest. Advantageously, a single temporary recovery VM, created basedon the VM template, can be used to restore multiple production VMs, dueto the established similarity between those production VMs.

When the production VM(s) have been restored 710 at the datacenter, thetemporary recovery VM may then be shut down 712. If any incrementalbackups of the VM full image have been generated and stored subsequentto creation of the VM full image, those incremental backups, ordifferentials, may be applied 714 to the restored production VMs so thatthe restored production VMs accurately reflect the configuration, bothwith respect to data stored and disk structure, of the production VMsbefore they were impacted by the DR event. Once the restored productionVMs are fully updated, they can then be placed into service and run 716.In some embodiments at least, the restored production VMs reside, andare accessible, at the datacenter. However, the restored production VMscan reside at any other DR target and are not necessarily hosted at thedatacenter.

F. Example Computing Devices and Associated Media

The embodiments disclosed herein may include the use of a specialpurpose or general-purpose computer including various computer hardwareor software modules, as discussed in greater detail below. A computermay include a processor and computer storage media carrying instructionsthat, when executed by the processor and/or caused to be executed by theprocessor, perform any one or more of the methods disclosed herein.

As indicated above, embodiments within the scope of the presentinvention also include computer storage media, which are physical mediafor carrying or having computer-executable instructions or datastructures stored thereon. Such computer storage media can be anyavailable physical media that can be accessed by a general purpose orspecial purpose computer.

By way of example, and not limitation, such computer storage media cancomprise hardware storage such as solid state disk/device (SSD), RAM,ROM, EEPROM, CD-ROM, flash memory, phase-change memory (“PCM”), or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other hardware storage devices which can be used tostore program code in the form of computer-executable instructions ordata structures, which can be accessed and executed by a general-purposeor special-purpose computer system to implement the disclosedfunctionality of the invention. Combinations of the above should also beincluded within the scope of computer storage media. Such media are alsoexamples of non-transitory storage media, and non-transitory storagemedia also embraces cloud-based storage systems and structures, althoughthe scope of the invention is not limited to these examples ofnon-transitory storage media.

Computer-executable instructions comprise, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions. Although the subject matter has been described inlanguage specific to structural features and/or methodological acts, itis to be understood that the subject matter defined in the appendedclaims is not necessarily limited to the specific features or actsdescribed above. Rather, the specific features and acts disclosed hereinare disclosed as example forms of implementing the claims.

As used herein, the term ‘module’ or ‘component’ can refer to softwareobjects or routines that execute on the computing system. The differentcomponents, modules, engines, and services described herein may beimplemented as objects or processes that execute on the computingsystem, for example, as separate threads. While the system and methodsdescribed herein can be implemented in software, implementations inhardware or a combination of software and hardware are also possible andcontemplated. In the present disclosure, a ‘computing entity’ may be anycomputing system as previously defined herein, or any module orcombination of modules running on a computing system.

In at least some instances, a hardware processor is provided that isoperable to carry out executable instructions for performing a method orprocess, such as the methods and processes disclosed herein. Thehardware processor may or may not comprise an element of other hardware,such as the computing devices and systems disclosed herein.

In terms of computing environments, embodiments of the invention can beperformed in client-server environments, whether network or localenvironments, or in any other suitable environment. Suitable operatingenvironments for at least some embodiments of the invention includecloud computing environments where one or more of a client, server, orother machine may reside and operate in a cloud environment.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed is:
 1. A non-transitory computer readable storage mediumhaving stored therein instructions which are executable by one or morehardware processors to perform operations comprising: receiving a fullimage backup of a virtual machine (VM) template that comprises data anda disk structure, and each virtual machine (VM) in a cluster that wasused to create the VM template comprises a defrag agent operable toensure that data and a disk structure of that VM match the data and thedisk structure of the VM template; storing the full image backup of theVM template; receiving a disaster recovery (DR) request that identifiesone or more target virtual machines (VMs) to be restored; creating atemporary recovery virtual machine (VM) based on the full image backupof the VM template; using the temporary recovery VM to restore one ormore of the target VMs, and one of the restored target VMs has the samestored data and disk structure as that one restored target VM had priorto occurrence of an event that was the basis for the DR request; andapplying any differential backups to a target virtual machine (VM) thathas been restored.
 2. The non-transitory computer readable storagemedium as recited in claim 1, wherein the operations further compriseshutting down the temporary recovery VM after the one or more target VMshave been restored.
 3. The non-transitory computer readable storagemedium as recited in claim 1, wherein the operations are performed at acloud datacenter.
 4. The non-transitory computer readable storage mediumas recited in claim 1, wherein after restoration, the restored targetVMs are accessible at a cloud datacenter.
 5. The non-transitory computerreadable storage medium as recited in claim 1, wherein the target VMsare selected as target VMs based on their similarity to each other. 6.The non-transitory computer readable storage medium as recited in claim1, wherein the particular full image backup of the VM template that isused in the creating is selected based on a particular restore point ortime identified in the DR request.
 7. A method, comprising: receiving afull image backup of a virtual machine (VM) template that comprises dataand a disk structure, and each virtual machine (VM) in a cluster thatwas used to create the VM template comprises a defrag agent operable toensure that data and a disk structure of that VM match the data and thedisk structure of the VM template; storing the full image backup of theVM template; receiving a DR request that identifies one or more targetVMs to be restored; creating a temporary recovery VM based on the fullimage backup of the VM template; using the temporary recovery VM torestore one or more of the target VMs, and one of the restored targetVMs has the same stored data and disk structure as that one restoredtarget VM had prior to occurrence of an event that was the basis for theDR request; and applying any differential backups to a target VM thathas been restored.
 8. The method as recited in claim 7, furthercomprising shutting down the temporary recovery VM after the one or moretarget VMs have been restored.
 9. The method as recited in claim 7,wherein after restoration, the restored target VMs are accessible at acloud datacenter.
 10. The method as recited in claim 7, wherein thetarget VMs are selected as target VMs based on their similarity to eachother.
 11. The method as recited in claim 7, wherein the particular fullimage backup of the VM template that is used in the creating is selectedbased on a particular restore point or time identified in the DRrequest.
 12. A system, comprising: one or more hardware processors; anda computer readable storage medium having stored therein instructionswhich are executable by the one or more hardware processors to performoperations comprising: receiving a full image backup of a virtualmachine (VM) template that comprises data and a disk structure, and eachvirtual machine (VM) in a cluster that was used to create the VMtemplate comprises a defrag agent operable to ensure that data and adisk structure of that VM match the data and the disk structure of theVM template; storing the full image backup of the VM template; receivinga DR request that identifies one or more target VMs to be restored;creating a temporary recovery VM based on the full image backup of theVM template; using the temporary recovery VM to restore one or more ofthe target VMs, and one of the restored target VMs has the same storeddata and disk structure as that one restored target VM had prior tooccurrence of an event that was the basis for the DR request; andapplying any differential backups to a target VM that has been restored.13. The system as recited in claim 12, wherein the operations furthercomprise shutting down the temporary recovery VM after the one or moretarget VMs have been restored.
 14. The system as recited in claim 12,wherein the operations are performed at a cloud datacenter.
 15. Thesystem as recited in claim 12, wherein after restoration, the restoredtarget VMs are accessible at a cloud datacenter.
 16. The system asrecited in claim 12, wherein the target VMs are selected as target VMsbased on their similarity to each other.
 17. The system as recited inclaim 12, wherein the particular full image backup of the VM templatethat is used in the creating is selected based on a particular restorepoint or time identified in the DR request.